In the industrial environment, digital cameras are often used to monitor production processes, machines or objects. Such industrial cameras are used for simple monitoring functions and for metrological tasks such as quality control by image processing, for example. They are distinguished by their ability for integration into arrangements and their high image quality.
Bayer sensor is the name of a sensor which—similar to a checkerboard—is covered by a color filter, which in most cases consists of 50% green and 25% red and blue each. In doing so, it is considered that the human eye reacts more sensitive to green than to other colors. Many conventional CCD sensors in digital photo and film cameras apply the concept of the Bayer matrix (or Bayer pattern). The “Bayer” matrix or “Bayer” filter is named after the inventor, Bryce E. Bayer, and is explained in more detail in the U.S. Pat. No. 3,971,065.
Light-sensitive cells of a semiconductor material of an image sensor can detect luminescence values only. To obtain color information, a tiny color filter in one of the three basic colors red, green or blue is arranged in front of each single cell. The filters are applied in the order green-red-green-red in odd rows and in the order blue-green-blue-green in even rows, for example. Each light-sensitive cell (pixel) of the image sensor, which has such a filter, correspondingly delivers information for a single color component at this position only. However, to obtain a full color image, a color value is required for each pixel, which consists of several color components, red, green and blue, for example. If a full color image having the same dimensions and the same resolution is to be generated, interpolation of missing color information from values of neighboring pixels is required. Thus, 50% of the green values of a color image are calculated, for blue and red it is 75% of the color values each (or, in one row 50% and in the next row 100% of the row), which are to be filled by the calculation. This interpolation is based on the assumption that only little color differences occur between neighboring pixels in the image. However, naturally this does not apply to each image subject. Therefore, strictly speaking, the Bayer sensor offers only a quarter of the virtual resolution taking into consideration a color image free of artefacts.
FIG. 1 shows an exemplary application of a digital industrial camera. Digital industrial cameras 10 are provided for integration in an industrial apparatus 23, as part of which they detect images of objects 14, 15 or human beings, for example. These images are transferred as image data 16, so that they can be evaluated and/or archived. The evaluation is often done automatically by an image processing means 18. Often, automatized actions are performed in response to the evaluation results, which are in context with the apparatus 23. In this way, bright objects 14 and dark objects 15, which can be discriminated by their color, are automatically sorted based on optical criteria. Such sorting is usual in various industrial sectors. As an example, food, e.g. grain according to the criterion of being free from stain, pieces of ore and rock based on their color or their brightness, postal items based on the address arranged on them, brake discs and sealing rings based on their compliance with a correct shape and their compliance with specific nominal sizes, or empty returnable bottles based on a correct shape and the barcode arranged on them. Such sorting often serves for quality checking and assurance.
Such industrial apparatuses 23 often comprise driving components which feed the objects 14, 15, e.g. a feeding belt 11 with a driving motor 12 and a feeding rate 13. The camera 10 records the image at a suitable position by means of an optics 22, such as an object lens. Thereby, the camera 10 may be arranged as area-scan camera, as row-scan camera or as multiple-row-scan camera, as described in the EP 1 919 197 A2, for example. The camera forwards image signals 16 of the recorded image to the image processing means 18 (e.g., a computer (C)). Furthermore, the image processing means 18 may optionally be supplied with an information 17 about a feeding rate 13 of the feeding belt 11, which may be determined e.g. by a rotary encoder. Thereby, the image processing means can determine a suitable point in time, at which an object passes an actuator 19 such as an element dropping past numerous controllable air injectors, and can supply a control signal 24 thereto, based on which an action is performed, which is suitable to lead the corresponding object to a sorting. As an example, the control of the air injectors can determine whether the object drops into a first container 20 if the airflow is switched off, or is deflected to a second container 21 if the airflow is switched on.
In such an application, it is advantageous if the camera 10 can transfer the image as fast as possible, so that the image processing means 18 can generate the control signal for the actuator 19 as fast as possible and can thus initiate the desired control action at an early stage. Thereby, the apparatus 23 can be operated faster. In certain applications, especially when the objects 14 and 15 move, e.g. roll, on the feeding belt 11, higher accuracy or position can be achieved or other parameters can be optimized.
In other application cases, recording and/or documentation of a circumstance may be of interest, as well. As an example, images of vehicles are recorded, which move disorderly in road traffic, e.g., when violating speed limit or passing red traffic lights—as described in the DE 10 2010 003 039 A1, for example. As an example, loading and unloading of ferryboats may also be accompanied by recording images of vehicles, to be able to determine whether a damage has occurred on the ferryboat in cases of claims for compensation. In such applications, it is required that the color of the objects and certain circumstances, e.g., red traffic lights, are reproduced correctly. Furthermore, it is important that the image has a good quality. This is the case, among others, if the objects of the image can be easily identified and letters, especially such with black and white contrast, are reproduced in a sharp and easily readable manner.
FIG. 2 shows a schematic representation of a structure of a digital camera 10 with an objective 22. An image scene 30 is reproduced via the objective 22 on an image sensor which comprises a regular arrangement of light-sensitive elements (pixels). The image sensor 31 transfers electronic data to a processing unit 32 mostly included in the camera 10, which comprises a processor, a digital signal processor (DSP) or a so-called Field Programmable Gate Array (FPGA), for example. In this case, it may be required that analogue image data is converted into digital image data, e.g., by means of an analogue-to-digital converter (AD-converter). In the processing unit, the image data is converted into a user-applicable form and is then output as an electronic signal 34 via an interface 33.
A user of such a digital color camera often wants to use a color image in which a full color value is available for each pixel position. Such a color value is then understood as a position or position in a color space, wherein the definition of the position in the three-dimensional color space is achieved by an indication of at least three components. The most common color spaces are three-dimensional, e.g. the RGB, sRGB, XYZ, YUV, and L*a*b* color space. Color values of one color space can be translated into color values of another color space, see for example A. Koschan and M. Abidi: “Digital Color Image Processing”, John Wiley & Sons, Hoboken 2008, ISBN 978-0-470-14708-5. Furthermore, methods for transmitting color values are known, e.g., as 8-Bit RGB, 12-Bit RGB or YUV 4-2-2.
As each pixel is sensitive for one color only, that is, it delivers one-dimensional information, determination of a color value for this pixel as a value in the three-dimensional color space is under-determined. Therefore, determination of missing information is commonly done by including signals of neighboring pixels, especially such pixels having color masks of another color. A corresponding calculation is called debayring, demosaicing, or color filter array (CFA) interpolation, and is described in the U.S. Pat. No. 6,181,376, for example. As a result of the debayring, a multidimensional color value is obtained for each pixel.
Industrial cameras are known, in which debayring is performed. Also, cameras are provided, which do not transmit images as color values, but as raw data, to a computer. Then, it is required that debayring is performed in the computer. Here, the debayring can be performed on a plug-in card used for receiving image data, a so-called Frame Grabber, or by a processor of the computer or the graphics board. There are also cameras for which a so-called software-implemented driver is offered. It often contains a program code for executing the debayring. It is known that disadvantageous wrong colors may occur when debayring is performed, which is called color aliasing. This problem is extensively discussed in the US 20100085452 A, for example. A proposal for preventing such Color aliasing by a so-called anti-aliasing is described in the U.S. Pat. No. 6,819,801, among others.
For the performance of a digital (industrial) camera, especially speed and image quality are of vital importance. The latter is determined according to the standard EMVA 1288 (www.emva.org) in the current version 3.0. Furthermore, subjective human perception of the image is used by the user as important criterion for judgement of image quality.
Digital industrial cameras are applied in different industrial sectors, e.g., food industry, logistics, transport, automotive industry, robotics and others. Within these industrial sectors there are many further different applications with different requirements, e.g., with respect to the size of the image sensor, the optical resolution given by the number of the pixel, with respect of the repetition rate measured in images per second (fps), and with respect to the data transmission format, e.g., according to the standards CameraLink, IEE 1394, GigE Vision or USB. Digital industrial cameras are known, which achieve high data rates, such as for example CameraLink with up to 800 Mbyte per second or GigE Vision with up to 100 Mbyte per second. To ensure efficient production at such a variety of requirements, such cameras are often arranged modularly, e.g. comprising a sensor module having an image sensor, a processing module and an interface module with an interface. Since the processing module is required to adapt a plurality of different sensor modules and interface modules, it is helpful if it is configurable by software, whereby the variety of types of processing modules and the related effort for development, production and logistics can be reduced. Furthermore, it is required to achieve high data throughput to thereby obtain the high required data rates. In this case, the processing module is often realized by an FPGA as central component.
The number of operations contained in a mathematical image processing method should be as little as possible. In case of an FPGA this leads to a low usage of logic cells, which is advantageous to allow usage of a cheap FPGA or to make available more free logic cells of the FPGA for other functions. In case of a DSP or another processor, a low number of operations means that these can be applied to a higher number of pixels within a given time span, whereby an image output at higher data rate is possible. Also, a low-power and thus cheaper processor can be used due to the lower number of image processing operations, and circuit complexity can thus be reduced.